Towards Flexible Multi-Agent Decision-Making Under Time Pressure

To perform rat ional decision-making, autonomous agents need considerable computat ional resources. In mult i-agent settings, when other agents are present in the environment, these demands are even more severe. We investigate ways in which the agent's knowledge and the results of deliberative decision-making can be compiled to reduce the complexity of decision-making procedures and to save t ime in urgent situations. We use machine learning algorithms to compile decision-theoretic deliberations into condit ion-action rules on how to coordinate in a mult i-agent environment. Using different learning algori thms, we endow a resource-bounded agent w i t h a tapestry of decision making tools, ranging from purely reactive to ful ly deliberative ones. The agent can then select a method depending on the t ime constraints of the part icular si tuat ion. We also propose combining the decision-making tools, so that , for example, more reactive methods serve as a pre-processing stage to the more accurate but slower deliberative decision-making ones. We validate our framework w i th experimental results in simulated coordinated defense. The experiments show that compil ing the results of decision-making saves deliberat ion t ime while offering good performance in our multi-agent domain. 1 I n t r o d u c t i o n It is desirable that an autonomous agent, operating under uncertainty in complex environments, be able to make opt imal decisions about which actions to execute. Rational decision-making under such circumstances using, for instance, the paradigm of expected ut i l i ty maximizat ion, is costly [Horvi tz, 1988; Russell and Wefald, 1991; Russell and Subramanian, 1995; Zilberstein and Russell, 1996]. In our work, we consider addit ional complexities presented by mult i-agent environments. In these settings, an agent has to make decisions as to the rat ional course of action considering not only the possibly complex and not ful ly known state of its environment, but also considering the beliefs, goals, intentions and actions of the other agents. Clearly, these demands may lead to its failure to decide an action wi th in the t ime constraint. To cope w i th t ime constraints imposed by various decision-making situations in complex and uncertain rnulti-agent settings, we endow an agent w i th a tapestry of decision-making procedures, f rom str ict ly reactive to purely deliberative. The reactive procedures are constructed by compil ing the deliberative decision-theoretic reasoning into condit ion-action rules. The compilation process exploits the regularities of the decision-theoretic reasoning and avoids costly deliberations in urgent situations. The rules are obtained from machine learning algori thms, which, as inputs, use the results of fu l l blown decision-theoretic computations performed offline. Each of the compiled methods is assigned a performance measure that compares it to the ful l-blown decision-theoretic benchmark. The various compilations available, and their combinations w i th more deliberative methods, constitute a spectrum of approaches to making decisions under the constraints of available computat ional (and cognitive) resources, and under t ime pressure. Given the various decision-making methods at its disposal, an agent should consider a number of factors to choose the appropriate decision-making mechanism for the si tuat ion at hand. The key factors include the quality of the decision provided by a method, the method's running time, and the urgency of the situation at hand. Intui t ively, when a si tuat ion is not urgent, the agent can afford the luxury of fu l l -b lown decision-theoretic reasoning since it results in highest qual i ty of the choice made. If the si tuat ion is very urgent, the agent should save as much t ime as possible by using a crude but fast react ive tool . If the si tuat ion is somewhat urgent, the agent should use methods that are somewhat sophisticated although not necessarily opt imal . Interestingly, the spectrum between the purely reactive and ful ly deliberative decision-making tools can be spanned by combining these two varieties of methods. For example, the agent can use fast reactive rules as a